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The bulk of this report consists of three handbooks and a special validation 
report, encompassing * the results of research , consultation with professional 
committees J and significant position stateiVients in four major areas of foreign 
language testing; placement of incoming students in foreign languages at the 
college level; the use of standardised achievement tests iu foreign languages and 
tlie preparation of teacher-made tests; interpretation of the results of the MLA 
Foreign Language Proficiency Tests for Teachers and Advanced Students; and 
validation of the Proficiency Tet;ts in terms of the absolute criterion > the 
native speaker. 

Part One, A Ha ndbook, o n Forel<3;n Lan^.uage Classroo ii) Testings French., Gorman^ 
Italian , Russl an , Spanish ireer.s the principal kinds of testing devices, 
considered useful in classroom teaching. Informative guidance is provided on 
the importance and place of testing in the foreign language program; planning 
the classroom test; the construct/Ion of test items; preparing test /items (including 
sample items for each of .the five languages), and the interpretation and use of 
test results. • . 

P a r t Tw o , A IL-andbook on the MIA Foreig n Lang ua)i ;e Profici ency Tests for Te achcrs:; 
and Advanced Students : Their Nature, Uses , a nd Limita tious , affords the prospective 
user of the MTA Proficiency Tests with a detailed and compreliensive view of tlie 
Tests, The rationale, qualification statements » and format of the tests are 
reviev/ed, sample items in all five languages are provided, and an item analysis is 
undertaken. Reports of four research projects particularly relevant to the use of 
the tests are described and a final chapter discusses the .uses and limitations of 
the tests . 

Part Three, A H andbo ok on Placement in For eign Langua^^e s, is based largely on 
information developed from a study' of placement practices at fifteen colleges and 
universities. Assumptions underlying current placement practices are .analysed , 
and a new approach is proposed involving a number of suggested assumptions and 
techniques. A set of appendices provide additional insights and views. 

A separate report presents the results of a project designed to validate the MLA 
Proficiency Tests in terms of the absolute criterion , the native speaker. The title 
of the report is The±i]J^^Jgv Te sts for Teachers and Advanced 

Students: Analysis of the Perform a nce' of Native Speakers and Comparison with that of 
NDEA Sum mer Institute Participants . . 
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Objectives : The Modern Language Association (MLA) proposed 
to provide professional leadership in the field of, foreign 
language testing in four areas: more efficient placement of 
incoming students D.n foreign • languages at the college level; 
better use of standardized achievement tests in foreign 
languages and more sophisticated preparation of toacher-Tiade 
foreign language tests; validation of the hT.A Foreign Language 
}?rof ic.iency Tests for Teachers and Advanced Students (rrofi- 
ciency Tests) in terms of .the absolute criterion, the native 
.speaker; more sophisticated interpretation of the results of 
the Proficiency. Tests. 



Procedures : Undei: the guidance of an Advisory Committee, the 
MLA conducted the following activities. (1) A conference on 
college foreign language placement was held- with the cooperation 
of fifteen college foreign language department chairmen and the 
College Entrance E^^amination Board, From the conference work- 
papers cind a series of case studies, a handbook on college 
foreign language placement was developed. (2) Modern foreign 
language m.ethods instructors, ' measurements specialists, and 
language teachers participated in a series*"' of conferences on the 
construction of classroom foreign language tests. After these 
meetings and field trials, a handbook on the construction and ■ 
use of foreign, language tests V7as written. (3) In order to 
validate the Proficiency Tests, the bILA administered them to 
native speakers living in their country of origin,, (4) llie MLA 
Director of Testing, his staff and the Foreign Language Test 
Advisory Committee continued to provide professional supervision 
in ,t:he use of the Proficiency Tests and. Educational Testing 
Ser;vice (ETS) in their administration of the program to insuye 
that it is consistent with public and professional interest* 

Results : Three handbooks were produced: A Handbook on Place- 
ment in Foreign Langua ges in Colleges and Universitie s; A Hand- 
book on the MLA FL Proficiency Tests for Teachers and Advanced 
Students: Their Nature, Uses^ and Limitations ; and A Measure- 
ments Handbook for Teachers of French, German, Italian^ Russian, 
and Spanish: A Guide to the Construction and Use of 'Foreign . 
Lan guage Tests. A. report entitled The MLA Foreign Language 
Prof iciencyvfests for Teachers and Advanced Students: Comparison 
of the Perid'rmance of Native Speakers .with that of NDEA Summer 
Foreign Language Institute Participants was published. 
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Contributio n t:o Hducal::! on : Through the above activities, the 
MLA has provided a continuing prograin of l.eadorfjhiip in foreign 
language nesting. These activities should strengthen, existing 
tests in foreign languages, assist Anerican teachers in properly 
using the standardised foreign "language tests, and educate them 
in the principles of solid test building. All of these activities 
vjill lead to much greater precision in foreign language testing, 
providing thereby improved measurement controls for modern foreign 
language instruction and basic and applied r'esearch in this area, . 
of American edvication. 



II • INTRODUCTION: BACKGROUND FOR THE STUDY 

^* General Statement 

. Through the activities supported by this contract the MLA 
proposed to provide professional leadership in the field of 
foreign language testing. Four needs were seen ' as especially 
urgent: the assessment of the foreign language resources of 
the nation; the assessment of the need for the development of 
nev7 tests and improved forms of e>d.sting foreign language' 
tests; the promotion of effective use of test results in order 
to improve instruction in modern foreign languages in schools 
and colleges; improved measurement in research in foreign 
language teaching. 

In designating the above activities as important, the MIjA 
reflected the increasing concern of members of the profession 
V7ith foreign language testing. Inquiries sent to the MLA asking 
for assistance in the selection and use of foreign. language tests 
and test information, publications by such professional groups as 
the Northeast Conference on the Teaching of Foreign Languages, 
and evaluations of foreign language tests,. such as those in the 
Sixth Mental Measurements Yearbook ,^ all testified to an expanding 
need for leadership in foreign, language testing. 



B. Related Research * . , 

The HLA developed the Proficiency Tests under 'the authority 
of the United States Office of Education (USOE) Contract LJo. SAE 
83^i9, ^tfatcd 11 June 1959. Under subsequent contracts (OE-4-14-035 
and 0E^6- 14-002) the MLA performed services related to the Profi- 
ciency Tests, and under Contract No. SAE 8827 it developed the MLA 
Cooperative Foreign Language Tests (Cooperative Tests). 

Since their development, the Proficiency Tests have been 
administered by the Educational Testing Service (ETS) in Princeton, 
Nevj Jerv^ey; copies of tlie Cooperative Tests are sold by the 
^ Coopera.tivc Test Division of ETS. 

ERIC 
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Tbe Proficiency Tests are used by se\'eral sfates as part of 
their certification programs, by NDEA Foreign Language Institutes 
as part of their pre- and post-institute evaluation of participants^ 
and by many pre-service; programs as part of their evaluation of 
future foreign language teachers. The Cooperative Tests are used . 
extensively as diagnostic, achievement, and placement instruments, 
in secondary schoo'lri:, colleges, and universities* Both sets of 
tests liave been used in basic and applied research by such members 
of the profession as Prof essor John' Carroll, Harvard Univer5jity, 
and Dr, Richard Spencer , Measurement and Research Division, 
University of Illinois. , Under Contract ■OE--6-14'-'002 , the HLA con- • 
ducted a study designed to "vertically equate the Cooperative 
Tests to the Proficiency Tests, and a project to evaluate the 
Proficiency Tests. 



C, Problems ' • ' 

The four problems identified and procedures proposed V7erG 
essentially those of the MLA Test Advisory Conmiittee whose members 
determined these to be especially crucial and. in need of immediate 
professional attention, 

1 • College Foreign Language Placement 

One of the perer.aial problems facing college departments 
of foreign languages is the proper placement of students who have 
begun study of a language in high school and find that the college 
department has expectations and approaches which differ consider- 
ably from those they have already encountered. Most often, college 
departments do not knov7 the foreign language background of entering 
stu.'r^nts while on' the other hand, high school departm.ents are' 
unaware of the standards and goals for foreign language study on 
the college level. Misplacement in college foreign language classes 
is a source of confusion for students, and can lead to disorientation, 
failure and eventual discouragement from continuing ].anguage study. 

The proper placement of students is further complicated by 
nev7' developments in foreign language teaching over the past years'. 
In the past the skills of reading and listening v;ere emphasized 
and, for most pre-college students, only a two or three year language 
sequence was available. Today, all four skills are taught and many 
school systems offer significantly longer language sequences, often 
begimiing in the elementary schools. Since each system has developed 
its o\vTi type of foreign language program the background of entering 
freshmen differs widely. The "Guidelines for Teacher Education 
Programs in Modern Foreign Languages," endoued by botli the MLA and 
the National Association of State Directors of Teacher Education and 
Certification (NASDTEC) , reflect the concern of the profession for 
good college placement programs in foreign languages. ^ 

The MLA frequently receives letters requesting information 
on co].lege foreign language placement. At its February 1966 ?neeting, 
the MLA Testj Advisory Committee stressed tlie need ^f or the profession 



to exainine, college placement: practices-. The College Entrance 
Examination Board indicated its concern by agreeing to participate 
with the MLA in a conference to discuss college placement procedures. 

2 . Educating the Foreign Language Teach i ng Profession in the 
Construction and Use o f Tests . 

In generals foreign language teachers at all levels of the 
educational system are both uninformed and unsophisticated about 
tests and testing. Their home'-produced tests all too frequently 
fail to isolate the specific learnings v/hich they seek to assess. 
Their interpretations of tlie results of commercially produced tests 
are likely to be naive. V7e are faced vjfith a major problem in 
attempting to inform the profession about this critical aspect of 
language teaclung. 

The information needs to be disseminated through various y 
channels. lastitutes designed to improve teachers already in 
service might well give more attention to testing in the courses 
devoted to teaching methods. The same v7ould be true of the current 
pre-service courses. Foreign language supervisors and school 
administrators generally could well profit from the same kind of 
information. 

If this information is to be disseminated, materials must 
be provided. An examination of existing materials rf:\reals, on the 
one hand, treatment so sophisticated or lengthy that it is inacces-- 
sible to many members of the profession, or, on the other hand, 
treatment so cursory as to imply that measurement is a relatively 
unimportant aspect of. the instructional process,- In order to 
remedy this situation, the MLA proposed to develop a set of inform-- 
ative materials for use in v7orkshops, KDEA Foreign Language 
Institutes, and methods courses. Building on existing materials 
concerned with testing in general, the MLA Handbook concentrated 
on the specific problems. of testing in modern foreign languages, 

3 . Val.idation of the Proficiency Tests: Administration to 
Native Speakers | 

'^Inasmuch as a ' superior ' level of competence in reading, 
V7riting, aural comprehension, and speaking was defined by the 
Steering Committee of the Foreign Language' Program of. the Modern 
Language Association of America as proficiency approximating 
that of an educated native, it is regrettable that the tests of 
the aforementioned abilities could not have been administered to 
a representative sampling of educated natives . * 

VJhile the Qualifications Statement^ for foreign language 
teachers includes references in various places to proficiency 
comparable to that pf "educated native speakers," the Proficienc}' 
Tests, which are based on that statement, had never been administered 
to native speakers in a. controlled , systematic project designed to 
ascertain the validity of the tests. Nor was there an explicit 
statement of v;hat the native speaker is assir^ied to knov7, either in 
terms of general competence, or in terms of performance on a 
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particular UasU. It was proposed / therefore, that data of the. 
latter kind be obtained by administering the Proficiency Tests 
to an appropriate sample of native speakers. 

The Test Advisory Committee reccmTriende'd that the MLA 
administer the tests to natives living in their country of origin 
and possessing at least a high school education. 

The result's of the project yielded general and specific 
■insights into the testing of foreign language skills and the 
validity of the Proficiency Tests, PerforDiance of native speakers 
was compared vjith that of non-native teachers and students, and 
items on the Pro|ficiency Test xv^ere identified v,-hic.h clearly 
confused or misled the native speakers'. 

4 . Professional Leadership in the Use of the Proficie ncy 

Tests ' " ' '. 

The MLA with the assistance of the USOE has maintained' 
supervision over the use of the Proficiency Tests since their 
development. Tiiere v/ere urgent reasons v/hy the l^.A should continue 
to do so : ' 

\ - . 

•a. Under its Contract No. OE~6-14-002, and in cooperation 
with ETS, the MLA recoimuended and helped to impleiiient a major change- 
in the administration -of the Proficiency Tests: starting 2 April 
1966. they vjere administered through- a National Testing Program 
(fixed dateSj fixed test centers, fixed testing options, etc.) 
rather than' through an institutional program (a system whereby an ■ 
institution or agency may order any test, at any time^ for adminis- 
tration at its convenience, and solely for its .candidates) • It vras 
necessary that some professional organization observe and analyze 
the impact of such a major change to assure "that it was consistent 
vvrith the professional and public interest. National administrations 
were given in April, September, and November of 1966, April, Sep- 
tember, and November of 1967, and in March of 1968. Tliis system v/as 
carefully observed for two years, and comprehensive recommendations 
lor altering the program were made in November of 1967 • 

b. Increased Aise by a greater variety of agencies has 
been accompanied by an increase in the number of requests for pro- 
fessional advice and assistance in using the test results. This 
public and professional interest continues to grov/, the change to 
the National Testing Program is likely to encourage use of^ the 
tests. . Professional advice to users of the Proficiency Tests has 
remained one of the needs of the public and profession, requiring 
specialixed materials for interpretation of test results, and 
personnel capable of advising institutions and agencies 3.n arriv- 
ing at informed and effective policies for using those results. 

t 

c. Since the tests were developed to *^T^aintai^ a con- 
tinuing surve}^ of the professional foreign language resources in 
teaching in the nation,*' and since it has always been the under- 
standing of those involved in supporting, developing, and adminis- 
tering the tests that the tests should reflect professional needs 
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and developments, it remains desirable that a responsible 
professional organization maintain constant communication with 
the profession and public. . This service includes providing. a 
forum to representatives of institutions and agencies to discuss 
the present state of the tests, the adcainistration of the program, 
and needed innovations in both. 

d. ETS has alv/ays sought and received the assistance of 
the MLA in recoBmicnding, screeninf-j and approving the professional 
scorers v;ho, arei needed to assure the high ].evel of reliability in 
this aspect of the program. Because of the high requi.remen ts and 
the natural turnover in personnel, there has. continued to be a 
need for such professional advice to ETS. 

e. It was imperative that a responsible professional 
organisation have the available personnel, facilities, and commu- 
nication with the profession to designate competent linguistics 
personnel to carry out revisions of existing forms of the tests 
and the development of new forms of the tests, as well as to make 
sure such revisions or new tests meet the specifications set 
forth by thejforeign . language profession. 



III. METHODS 



A. Colleji^e Foreign Language Pl ac ement 

At its October 1966 meeting j the MLA Test Advisory Committee 
drev7 up a list of types of institutions which should be represented 
in the placement project. \ letter was sent to foreign language 
coordinators and supervisors in all states , requesting information 
on what they considered the most serious placement problems and 
what issues they wanted to see treated in the MLA, handbook. Follow- 
ing the suggestions of the Test Advisory Comiuittee and the state 
supervisors and coordinators, the MLA invited fifteen chairmen of . 
foreign language departments, representing a broad range of state, 
private, small, and large colleges and universities , to participate 
in the Project on Foreign Language Placement. 

A conference of the fifteen department chairmen was lield in 
New York in March 1967 vzith a representative of the College Entrance 
Examination Board and the MLA Staff present. In preparation for the 
conference each chairman prepared a description of his institution 
and its actual foreign language placement practices. The descrip-- 
tions served as the basis for discvv^sions which led to a preliminary 
list of -placement problems, general recommendations for improvement 
of practices, and specific suggestions for the handbook. 
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Follow--up visits v^ere made by the MI.A Director of Testing, 
an MI^A Research Assistant and a representative of the CEEB to 
the campuses of the participating c]iairm(;in in April and May 1967, 
The purpose of the visits was to validate and expand the infor- 
mation presented in the chairmen's description through structured; 
interviev;s V7ith selected instructors aiid students. All three 
members of the project staff used the same guides for interviews 
and follov/ed comparablle ratios of students and Instructors on all 
campuses. Where available,., standarir^ed test data on participating 
students, and all other students, were studied. 

Throughout the period of the piroject, at professional meetings 
^ and through correspondence, the project staff solicited identifica- 

tion of problems and proposed solutions relating to the broad issue 
of continuity in foreign language study. 

The l^Lk Handbook v;as produced, based on. the three^ major phases 
of the project; 1) the results of the conference of chairmen, 2) 
the results of the on-ca.mpus interviews, and 3) the written respv^.ise 
to the direct solicitation for advice. 



Educating the Forei^^^n Language Teaching Profession in the 
Construction and Use of Tests ' . 

Follov7ing the suggestions, of. the Test Adviso^i-y Committee and 
others in the profession, the MLA invited fifteen modern foreign 
language teachers (three in each of the five commonly taught 
languages), three methods teachers, a measurements specialist, 
members of the Test Advisory Committee, and representatives from 
ETS.and USOE to a conference on 17-18 February 1967 to discuss 
the handbook on the construction and use of tests in foreign 
languages. The MLA provided each participant with background 
materials on testing in general, foreign language testing, and 
measurement for tlie non-specialist, - 

The participants discussed the need for the handbook, what . 
they thought the content and materials included in the handbook 
should be and^ the best approach to use to ensure that school 
teachers would read the handbook by achieving the best balance 
of theoretical and practical materials. The participants, 
working in language groups, produced several outlines. The 
writing of the handbook and the various parts of the handbook 
were also discussed, : . 

After the February conference, the M^A worked to consolidate 
othe outlines suggested for the handbook, Coimiiissioned writers 
were engaged, and first drafts of most of the chapters were 
written, ^ 

^ A second conference involving the commissioned writers 
and the participants at tlie first conference V7as held in New York 
in May 1967, The participants suggested corrections and revisions 
of the first draft. Meeting in language groups they discussed the 
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collected sample items and tentatively classified the items 
according to a matrix ijrovided by one of the corainissioned 
v;riters. For each language, one person was appointed to edit 
and classify the items in that language. 

ETS reviewed the items submitted in each language "and 
provided a detailed analysis of each item with suggestions for 
improvement, where necessary. The heads of the language teams 
came to New York to review the items and the ETS criticisms. 
They selected, classified and revised the items to be included 
in the second draft of the handbook. The commissioned writers 
submitted their revised chapters for review by the MLA staff 
and one general commissioned writer. 

Every Director of. an NDFA Foreign Language Institute in 
1967 was invited, along with his staff and institute participants 
to criticize the second draft of the handbook. These criticisms 
were collated and summarized. 

In August 1967, a final conference of all participants, com- 
missioned writers, and the tlLA Test Advisory Committee v/as held. 
In light of the criticisms from the Institutes and the criticisms 
of the conference participants, revisions to be incorporated in 
the final. draft were suggested. Taking the recoimiiendations given 
at the conference into consideration, , the commissioned writers 
wrote the final draft of the handbook. 

C. Validation of the Proficiency Tests; Administration to 
Native Speakers 

The Director of Testing appointed five Research Associates, 
feach an expert in one of the five commonly taught foreign languages. 
All Research Associates adapted the Proficiency Tests to foreign 
language testing conditions translating test directions into the 
foreign language and modifying the test booklets and answer sheets. 
To assure that the translated and modified materials v/ere readily 
understandable to native speakers, each of the Research Associates' 
administered th"e modified tests to a sample of ten native speakers.. 

In I each foreign country , a foreign coordinator was hired to 
coordinate the administration, to procure the necessary number of 
examinees, to prepare the language laboratory , and to make all 
necessary arrangements for the administration of the tests. 
Building on their experience with the^j sample administration to ten 
native speakers, all Research Associates adapted the tests to 
specific foreign language conditions, |:translating the directions, 
simplifying the marking of answers, and modifying the tapes as , . 
necessary. Personal Data Sheets were jprepared in the target 
language asking for information on each 'examinee's background. 

c The administration of the tests were completed, .in all 
countries involved in the project except Russia V7here negotiations 
.could not be finalized. Edward. A^llenj .administered 300. French 
tests in Lyon, France;" Gustave Hathieii, 300 German itests in VJest 
Berlin, Germany; Salvatore Castiglione, 300 Italian tests in 




Florence; and Filomena del Olcio, 100 Spanish tests in Madpi^d, 100 
in Santiago, Chile, and 100 in Bogota, Colombia, Booklets and tapes 
were returned to EXS where the Speaking and Writing Tests were scored 
by professional scorers at ETS as were the two machine scored parts' 
of the tests, the Reading and Listening Comprehension Tests, The 
personal data siieets were returned" to the MLA to be analyzed. 

Each Research Associate submitted to :1LA a report describ- 
ing the procedures used, problems encountered, and personnel involved 
in the administration of the Proficiency Tests abroad. The Reports 
also included advice on limitations or special interpretations which 
should be placed on test data and the Research Associates' observa- 
tions and evaluation of the project, j 

A report describing the project, reporting the data, and analyz- 
ing the tests scores and the data, from the personal data sheets has 
been produced by the MLA,^ 



D . Professional Leadership in the Use of the Proficiency Tests 

1. Proficiency Test Handbook 

The MLA Director of Testing and the Research Assistant met 
with menibers of the ETS staff to discuss the specific features of 
the handbook including features specifically recoiumended by the MLA 
Test Advisory Conmiittsie. The Handboo k for the Law Schfaol Admission 
Test: Its Nature, Uses and Limitations , which vjas prepared at the 
direction of the Law School Admssion Test Council, ' was the example 
for the ^^LA handbook.' The handbook incorporated the results of ' 
several studies on t:he Proficiency Tests such as Dr, John Carroll's 
work in testing college foreign language majors, the MLA staff 
project to vertically equate, the Cooperative Teists to the Proficiency 
Tests, and the native speaker study. Material oh the nature, uses, 
potential, and limitations of the Tests are also included in the 
handbook. The handbook extensively describes each of the seven areas 
of language teaching competence measured by the tests, describes each 
subsection of that test and provides sample items in all five lan- 
guages as well as selected data used to analyze the reliability of 
the item in( its ovm terms and in terms of the total test from which 
it is taken (item analysis) . Normative data could not be included 
in the handbook but were provided as a. separate insert, thus assuring 
both maximum' flexibility and up-to-date information at all times. 
The MLA staff undertook the major writing of the handbook, while the 
ETS staff gave technical assistance, 

2, The Dissemination of Information on the Use of the 
Profici enc y Tests 

TheMLA Director of Testing continued to publicize the Pro- 
ficiency Tests and to advise institutions, agencies and individuals 
in the use of the Tests. A paragraph on the National Testing Program 
was included in the first number of Foreig^n Language Annals , a publir 
cation of the Modern Language Association, Over 60,000 copies were 



distributed to 1) recipients of state foreign language newsletters 
2) all foreign language teachers wlio belong to the' MI^A and are 
resident in the United States and 3) various other special mailing 
lists. Brochures on the Proficiency Tests were prepared each year 
and sent to all state foreign language- consultants, editors of 
state and national foreign language newsletters, methods teachers,^ 
and city coordinators of foreign languages. At professional meet- 
ings, the MLA Research Assistant distributed booklets on the Pro- 
ficiency and Cooperative Tests and answered questions on the use 
and nature of the two tests. During the 1967-68 National Testing 
Program, the' ETS called the administrations to the attention of 
chairmen of foreign language departments and heads of state depart 
ments of education, 

3 . Supervision of Scoring Center at ETS 

The WLA staff continued to screen and approve applicants 
for the Scoring Center at ETS. On 6. March 1967, the MLA Coordi- 
nator , of Testing and the MLA Research Assistant met with several 
people from various departments of ETS to discuss establishment 
of a set of procedures for dealing vrith the problem of recruiting 
and training scorers • By 15 May 1967 a; memorandum of agreement 
based on the discussions was prepared settixig forth a preliminary 
description of the general outlines of the V7ork involved in 
setting up the new system, and described the specific procedures 
to be employed* The memorandum was approved by the MLA on a one- 
year plan of operation. ETS dealt with the routine problems of 
securing and training scorers, according to the ETS-I*iLA memorandum 
of understanding. While. ETS consulted the MLA only in cases of 
irregularities or possible changes in the system, close surveil- 
lance v/as kept over the program during the trial period. The 
first annual review was scheduled to take place in April 1968. 



4. The National Testing Program 




- / 

The Proficiency Tests were administered oh- 17 September 
1966 to 326 examinees in thirty testing ' centers around the Unitled 
States. \\Tiile the number of registrants was considerably belovj 
that for the April 1966 (800) , the >G-.A Staff was encouraged by the 
response v?hich V7as far above that e>q3ected. Por^ the 19 November 

1966 administration, 789 were registered at forty-nine centers. 

For the 1967-68 National Testing Program, five dates were 
confirmed. Although there had been a drop in FL Institute enroll- 
ment, the MLA expected that two new developments would offset this 
decline: first, according to the USOE handbook for preparing 
institute proposals, directors could now require that Institute 
applicants submit Proficiency Test scores; and second, almost half 
the institutions listed in the MLA publication, Vacancies in 
College and University Departments of Foreign Languages for Fall 

1967 stated that Prof iciency /lest scores were desired for the 
applicants. However, by November 1967, it was necessary to cancel 
two of the five test administration dates scheduled for 1967-68 
because of the combined effects of increasing costs and continuing 
low candidate volume for the Proficiency Tests. Approval for this 
was granted by the United States Office, of Education, 
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^ • Response to Inc^ui.ries About the Proficiency Tests 

The MLA has continued to provide the necessary personnel 
and facilities to respond to any inquiries relating to the Pro- 
ficiency Tests. 

6. Test Advisory Committee 

To advise the Director of Testing and his staff, the MLA 
appointed a fseven member^ Test Advisory Coinmittee which met several 
times during the period of this contract. The members represented 
institutions of higher education and education agencies (e»g,, state 
departments of education) and provided the cliannels of communication 
betv7cen the >ILA and the profession and the public* The Committee 
served as the editorial board for all the materials produced as part 
of this contract. The Director of Testing called to the attention f 
of the Test Advisory Committee major areas of concern in foreign 
language testing. Their advice helped to assure the continuing 
effective assessment of the nation's foreign language resources • 
The MLA has served as a clearinghouse in matters of foreign language 
testing for the country. 



IV. RESULTS AI\^D FINDINGS 



In accordance ^^ith the provisions of this contract three handbooks 
were produced, 

A. A Handbook on Foreign Language Classroom Testing: French > 
German, Italian, Russian, Spanish 

This Handbook has been produced to help the classroom teacher 
make efficient use of tests. It discusses the main purposes tests 
can serve and treats the principal kinds of testing devices that 
seem to be especially useful for those purposes in the context of 
foreign language teaching. And it has a section (Chapter V) on the 
interpretation of test results. 

Much of what is contained in this Handbook is useful at all 
levels of instruction and with people of all ages. Some testing 
devices, however, are obviously not appropriate to children in 
elementary or even junior high school classes. The teacher should 
have no difficulty making a judicious choice. 

Most teachers have in fact had some formal instruction in 
their oi-m undergraduate education in the general principles of 
measurement, although very few classroom teachers seem to have liad 
any formal instruction in principles and methods of testing in the 
specific context of foreign language teaching. Much of the general 
theoretical discussion contained in the Handbook will be more or 
less familiar to the reader. It is included here for several 
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reasons: to make" the Handb ook a self-contained unit; to remind 
the reader of principles and approaches he may have forgotten; 
and to fill Ir^ gaps in the reader's background. 

This Handbook does not airi^ to be complete. For some teachers 
it will suffice, for others it will serve to inspire tliem to 
exercise their imagination and ingenuity. The Handbook discusses 
the following topics; The Importance and Place of Testing in the 
Foreign Language Program; Planning the Classroom Test; the Con-- 
struction of Test Items; Preparing Test Iteiiis (a section is 
Included with sample items for each of the five languages); and 
The Interpretation and Use of Test Results. 

Most of the sample test items included in this Handbook were 
reviewed by the Foreign Language Test Staff 'of the Educational 
Testing Service. 



B ♦ A llandbook on the MLA Foreign Language Proficiency Tests for 
Teaclprs and Advanced Students: Their Nature, Uses and 
Limitations 

— , f 

This Handbook V7as produced to help the prospective user of 
the >1LA Proficiency Tests acquire a more comprehensive view of the 
Tests and how they might better interpret and use the results 
obtained. 

The Introduction to the Handbook gives the reader a short 
summary of the background that led tb the development of the l^Lk 
Proficiency Tests. The Tests are extensively described by (1)^ 
identifying the area of competence or knowledge, (2) repeating 
the appropriate section of the Qualifications Statement^, (3) 
providing a brief description of each sub-section of that test, 
(4) providing sample items in all five languages and (5) presenting 
selected data used to analyze the reliability of the item — in its 
own terms and in terms of the total test from which it is taken; 
in short, item analysis r 

The Third Section of the Handbook describes the validity of 
the tests and four comprehensive research projects particularly 
relevant to the use of the tests. Described are the studies of; 

1. Melton and Myers (1964) who examined the relationship 
between scores on the Proficiency Tests and qualitative ratings 
given to about 3,000 NDEA Foreign Language Institute participants. 7 

2. A Comparison of the level of difficulty of the MLA Foreign 
Language Proficiency Tests and the MLA Foreign Language Cooperative 
Tests was made by F. A. Paguette, Suzanne Tollinger, Madeline 
Wallmark, R. E. Spencer,. and F. J* Churchill^S 

3. John Carroll's gtudy of The Foreign Language Attainments 
of Language Ma.jors in the Senior Year: A Survey Conducted in U. S. 
Colleges and Universities . 9 The major purpose of the study vjas to 

ERLC 
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assess* the overall levels of foreign language proficiency attained 
by senior class students "majoring*' or concentrating in foreign 
languages in colleges and universities of the United States and to 
identify factors strongly associated with these levels of competence, 
with the hope such information rnight suggest ways in which foreign 
language teaching for these students could be improved. 

^. The analysis of the Performance on the Proficiency Tests 
of Native Spe^ikers and comparison with tliat of NDEA SuMier Institute 
Participants vms conducted by F. A. Paquette, SuEanne Tollinger and 
Madeline VJallinark.-^^ This validation of the Proficiency Tests by 
administering them to native speakers v;as performed under the terms 
of this contract. 

The final chapter discusses the uses of the tests and their 
potential limitations. The principle vjhich guided the design and* 
development of the Proficiency Tests and the nature of the tests 
themselves gives a much wider potential for the use of these tests 
than originally envisioned. Because there are seven specific sub- 
tests for each of the five commonly taught languages (and three 
alternate forms for French, German, Russian and Spanish and two 
alternate forms in Italian) there is great versatility in the 
batteries themselves. The major limitation pointed out in the 
Handbook is the fact that the examinations are not capable of 
measuring tue ability of any individual to function as a class- 
room teacher. 

The tests provide the direct functions of measuring a person's 
specific competence iii each of the individual skills and the three 
major areas of knowledge. It can measure proficiency prior to a 
particular experience and at tlie conclusion of it in order tp 
measure the effects of the experience • Indirectly they function 
in placement, including the screening of candidates for particular 
positions and the grouping oi students for any specific purpose. 

One of the most promising potential uses of the Tests is in 
the area of staff training; at least three specific possibilities 
present themselves: 

1. Through the use of the tests and other professional 
materials, a trainer of teachers, a department chairman, or any 
leader of foreign language teachers could educate his staff to 
write both performance objectives as well as general objectives 
for a foreign language program — either in the four skill areas or 
in the three knowledge areas* Individual items can become the 
basis for the development of specific performance objectives. 

2. Using the MLA Proficiency Tests in conjunction with 
Chapter III (Construction of Test Items) of the ^ILA Handbook on 
Foreign Language Classroom Testing , trainees could be presented 
with a much broader sampling of test items — by language and by 
skill or by knowledge area. The traiiiees could examine profes- 
sionally prepared items and then prepare items of their own based 
on these models. Of particular promise are the Listening Compre- 



hension and Speaking Tests — .^reas v/liich have always ppsed difficulty 
for individual teachers in the construction of testing instruments.' 

3. As is pointed out in the Handbook on Foreig n Lang uage Class- 
room Testi ng;, two of the most difficult areas of competence for 
teachers to evaluate are speech and writing because tliese involve so- 
called ^'free response" or "essay- type" questions. • Experience has 
sho\m that the halo effect involved in evaluating speaking and 
writing tests is th-a mokt difficult roadblocl: to effective assessment 
in these two skill areas* Through tlic use of the appropriate sub- 
tests in the- ITLA battery, a staff development prqigram in an individual 
scl^ool, school system, . college or university could be designed to 
train teachers in the objective scoring of oral and written essays. 

Through the use of tests developed by leaders of the foreign 
language teaching profession and measurement specialists, it is 
possible to train teachers to more properly use t!est results or to 
be much more sophisticated in the development and use of teacher-made 
tests • 

One major national use for the Proficiency Tests can be envisioned 
since the "secure tests" phc^se has concluded and since they also have 
been available for purchase for several years* One' can foresee their 
possible maximum use in the future as making up the core of a large 
data bank of individual test items in each of the languages and each 
of the knowledge areas. Thus, the profession could have available 
to it the basis for creating a pool for foreign language test items 
which could be used in a wide range of circumstances by school, college^ 
and other institutional personnel* In short, the trend towards per-- 
formance criteria and criterion reference tests could be considerably 
hastened by the availability of such a pool. The only action which is 
required to create this pool is a major decision on tlie part of the 
Modern Language Association of America. This may well be the greatest 
contribution yet to be made by the MLA Testing Program vchich began in 
1959. 



C. A Handbook on Placement in Foreign Languages in Colleges and 
Universities 

This project was designed to help the foreign language teaching 
profession find ways to improve continuity and articulation of in- 
struction from one grade level to the next and from one administrative 
unit to another. The Handbook is based on information gathered from 
a close study of the foreign language placement practices of fifteen 
colleges and universities, as well as from statements and publications 
gathered from the profession at large. 

Emerging from a comparison of the divergent placement 
practices at the fifteen institutions, a set of fourteen basic 
assumptions were identified and analyzed by the project staff. 
Proceeding from this analysis of prevailing assumptions, the authors 
of the Handbook propose, in Part V, a new approach to college foreign 
language placement, based on: j 
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!• A nev; set of assumptions v;hicii all students and teachers 
accept* 

•2, A new approach to describing the background and determining 
the probable success of each student. 

3. A new approach to describing school and college foreign 
language offerings. 

^. Several general recommendations for developing a continu- 
ing basis for sound placement practices in the future. 

Each of the four components of the nev? approach is outlined in 
detail in Part V of the Handbook . A set of appendices provide infor- 
mation on techniques used to obtain data from tlie fifteen institutions 
stud'^ed in detail, plus a selection of publications and position 
papers received from meinbers of the profession at large. 



V. CONCLUSIONS 



During the first half of the 1960'sj an Increase in prosperity 
and student population, together with a general awakening of interest 
in foreign language study, led to a rapid expansion of language In- 
struction at all levciils of American education. In many ways the 
foreign language teaching profession was unprepared for this expanded 
demnd for its '^product,** not only in terms of numbers, but also 
because the "product" itself had been widely "advertised" to be some- 
thing not all teachers were prepared to supply* The shift in 
emphasis ftom the traditional teaching of grammar and ti'anslation to 
speaking and listening skills had left many teachers and departments 
confused — and in many cases poorly informed — about aims and methods, 
techniques, and materials for audiolingual teaching. 

The area of testing and measurement was one in which the problems 
created by the "revolution" in foreign language teaching were most 
clearly noticeable, (1) Teachers were in desperate need of devices 
for evaluating student classroom performance according to criteria 
that would go beyond mere grammatical accuracy and employ as an 
ultimate standard the proficiency of a native speaker. (2) As more 
and more institutions adopted new methods and approaches in their 
language instruction, problems arose in connection vyith the place- 
ment of college freshmen who wished (or needed) to continue language 
study already begun in high school. College language departments 
recognized the need for objective measures of language proficiency 
unrelated to specific texts or teaching materials. (3) As it began 
to develop new criteria for measuring the proficiency of its 
students, the teaching profession also became aware of a need to 
evaluate the proficiency of its o\m practitioners and ultimately to 
standardize the training of future teachers. 
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In response to these needs, the Handboo k and other rai^Lerials 
prodiiced by this project liave as their aim nothing- less than thcj 
"profcssionali2:ation*' of tliose inenibers af the coTrununity of foreign 
language teachers in the United States who still fall short of 
prof ef^jfsional standards in their preparation or teaching practice, 
or both. Tlie suiterials will succeed only to the extent thej^ can 
be disseminated, discussed, and serioiisly irMpleinented by responsible 
members of the profession v?ho recogni:r:c the necessity of placing a 
common goal ahead of individual' convenience. Much depends upon the 
abilit/' of the leadership of the profession to arouse a sense of 
responsibility among the rank and file of foreign language teachers, 
individually, and collectively through their various organizations. 
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